Processing with Compact Arithmetic Processing Element

ABSTRACT

A processor or other device, such as a programmable and/or massively parallel processor or other device, includes processing elements designed to perform arithmetic operations (possibly but not necessarily including, for example, one or more of addition, multiplication, subtraction, and division) on numerical values of low precision but high dynamic range (“LPHDR arithmetic”). Such a processor or other device may, for example, be implemented on a single chip. Whether or not implemented on a single chip, the number of LPHDR arithmetic elements in the processor or other device in certain embodiments of the present invention significantly exceeds (e.g., by at least 20 more than three times) the number of arithmetic elements, if any, in the processor or other device which are designed to perform high dynamic range arithmetic of traditional precision (such as 32 bit or 64 bit floating point arithmetic).

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/399,884, filed on Feb. 17, 2012, entitled, “Processing with CompactArithmetic Processing Element,” now U.S. Pat. No. 8,407,273 (AttorneyDocket No. A0006-100° C.1); which is a continuation of U.S. patentapplication Ser. No. 12/816,201, filed on Jun. 15, 2010, entitled,“Processing with Compact Arithmetic Processing Element,” now U.S. Pat.No. 8,150,902 (Attorney Docket No. A0006-1001); which claims the benefitof U.S. Provisional Patent Application Ser. No. 61/218,691, filed onJun. 19, 2009, entitled, “Massively Parallel Processing with CompactArithmetic Element” (Attorney Docket No. A0006-1001L); all of which arehereby incorporated by reference herein.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND

The ability to compute rapidly has become enormously important tohumanity. Weather and climate prediction, medical applications (such asdrug design and non-invasive imaging), national defense, geologicalexploration, financial modeling, Internet search, networkcommunications, scientific research in varied fields, and even thedesign of new computing hardware have each become dependent on theability to rapidly perform massive amounts of calculation. Futureprogress, such as the computer-aided design of complex nano-scalesystems or development of consumer products that can see, hear, andunderstand, will demand economical delivery of even greater computingpower.

Gordon Moore's prediction, that computing performance per dollar woulddouble every two years, has proved valid for over 30 years and lookslikely to continue in some form. But despite this rapid exponentialimprovement, the reality is that the inherent computing power availablefrom silicon has grown far more quickly than it has been made availableto software. In other words, although the theoretical computing power ofcomputing hardware has grown exponentially, the interfaces through whichsoftware is required to access the hardware limits the ability ofsoftware to use hardware to perform computations at anything approachingthe hardware's theoretical maximum computing power.

Consider a modern silicon microprocessor chip containing about onebillion transistors, clocked at roughly 1 GHz. On each cycle the chipdelivers approximately one useful arithmetic operation to the softwareit is running. For instance, a value might be transferred betweenregisters, another value might be incremented, perhaps a multiply isaccomplished. This is not terribly different from what chips did 30years ago, though the clock rates are perhaps a thousand times fastertoday.

Real computers are built as physical devices, and the underlying physicsfrom which the machines are built often exhibits complex and interestingbehavior. For example, a silicon MOSFET transistor is a device capableof performing interesting non-linear operations, such as exponentiation.The junction of two wires can add currents. If configured properly, abillion transistors and wires should be able to perform some significantfraction of a billion interesting computational operations within a fewpropagation delays of the basic components (a “cycle” if the overalldesign is a traditional digital design). Yet, today's CPU chips usetheir billion transistors to enable software to perform merely a fewsuch operations per cycle, not the significant fraction of the billionthat might be possible.

SUMMARY

Embodiments of the present invention are directed to a processor orother device, such as a programmable and/or massively parallel processoror other device, which includes processing elements designed to performarithmetic operations (possibly but not necessarily including, forexample, one or more of addition, multiplication, subtraction, anddivision) on numerical values of low precision but high dynamic range(“LPHDR arithmetic”). Such a processor or other device may, for example,be implemented on a single chip. Whether or not implemented on a singlechip, the number of LPHDR arithmetic elements in the processor or otherdevice in certain embodiments of the present invention significantlyexceeds (e.g., by at least 20 more than three times) the number ofarithmetic elements in the processor or other device which are designedto perform high dynamic range arithmetic of traditional precision (suchas 32 bit or 64 bit floating point arithmetic).

In some embodiments, “low precision” processing elements performarithmetic operations which produce results that frequently differ fromexact results by at least 0.1% (one tenth of one percent). This is farworse precision than the widely used IEEE 754 single precision floatingpoint standard. Programmable embodiments of the present invention may beprogrammed with algorithms that function adequately despite theseunusually large relative errors. In some embodiments, the processingelements have “high dynamic range” in the sense that they are capable ofoperating on inputs and/or producing outputs spanning a range at leastas large as from one millionth to one million.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example overall design of a SIMD processor according to oneembodiment of the present invention.

FIG. 2 is an example of the Processing Element Array of a SIMD processoraccording to one embodiment of the present invention.

FIG. 3 is an example of how a Processing Element in a Processing ElementArray communicates data with other parts of the processor according toone embodiment of the present invention.

FIG. 4 is an example design for a Processing Element according to oneembodiment of the present invention.

FIG. 5 is an example LPHDR data word format according to one embodimentof the present invention.

FIG. 6 is an example design for an LPHDR arithmetic unit according toone embodiment of the present invention.

FIG. 7 is an original image.

FIG. 8 is an image blurred by a blur kernel according to one embodimentof the present invention.

FIG. 9 is an image produced by Richardson Lucy deconvolution usingfloating point arithmetic according to one embodiment of the presentinvention.

FIG. 10 is an image produced by Richardson Lucy deconvolution usingLPHDR floating point arithmetic with added noise (fp+noise) according toone embodiment of the present invention.

FIG. 11 is an image produced by Richardson Lucy deconvolution usingLPHDR logarithmic arithmetic (Ins) according to one embodiment of thepresent invention.

DETAILED DESCRIPTION

As described above, today's CPU chips make inefficient use of theirtransistors. For example, a conventional CPU chip containing a billiontransistors might enable software to perform merely a few operations perclock cycle. Although this is highly inefficient, those having ordinaryskill in the art design CPUs in this way for what are widely accepted tobe valid reasons. For example, such designs satisfy the (oftenessential) requirement for software compatibility with earlier designs.Furthermore, they deliver great precision, performing exact arithmeticwith integers typically 32 or 64 bits long and performing ratheraccurate and widely standardized arithmetic with 32 and 64 bit floatingpoint numbers. Many applications need this kind of precision. As aresult, conventional CPUs typically are designed to provide suchprecision, using on the order of a million transistors to implement thearithmetic operations.

There are many economically important applications, however, which arenot especially sensitive to precision and that would greatly benefit, inthe form of application performance per transistor, from the ability todraw upon a far greater fraction of the computing power inherent inthose million transistors. Current architectures for general purposecomputing fail to deliver this power.

Because of the weaknesses of conventional computers, such as typicalmicroprocessors, other kinds of computers have been developed to attainhigher performance. These machines include single instructionstream/multiple data stream (SIMD) designs, multiple instructionstream/multiple data stream (MIMD) designs, reconfigurable architecturessuch as field programmable gate arrays (FPGAs), and graphics processingunit designs (GPUs) which, when applied to general purpose computing,may be viewed as single instruction stream/multiple thread (SIMT)designs.

SIMD machines follow a sequential program, with each instructionperforming operations on a collection of data. They come in two mainvarieties: vector processors and array processors. Vector processorsstream data through a processing element (or small collection of suchelements). Each component of the data stream is processed similarly.Vector machines gain speed by eliminating many instruction fetch/decodeoperations and by pipelining the processor so that the clock speed ofthe operations is increased.

Array processors distribute data across a grid of processing elements(PEs). Each element has its own memory. Instructions are broadcast tothe PEs from a central control until, sequentially. Each PE performs thebroadcast instruction on its local data (often with the option to sitidle that cycle). Array processors gain speed by using siliconefficiently—using just one instruction fetch/decode unit to drive manysmall simple execution units in parallel.

Array processors have been built using fixed point arithmetic at a widevariety of bit widths, such as 1, 4, 8, and wider, and using floatingpoint arithmetic. Small bit widths allow the processing elements to besmall, which allows more of them to fit in the computer, but manyoperations must be carried out in sequence to perform conventionalarithmetic calculations. Wider widths allow conventional arithmeticoperations to be completed in a single cycle. In practice, wider widthsare desirable. Machines that were originally designed with small bitwidths, such as the Connection Machine-1 and the Goodyear MassivelyParallel Processor, which each used 1 bit wide processing elements,evolved toward wider data paths to better support fast arithmetic,producing machines such as the Connection Machine-2 which included 32bit floating point hardware and the MasPar machines which succeeded theGoodyear machine and provided 4 bit processing elements in the MasPar-1and 32 bit processing elements in the MasPar-2.

Array processors also have been designed to use analog representationsof numbers and analog circuits to perform computations. The SCAMP issuch a machine. These machines provide low precision arithmetic, inwhich each operation might introduce perhaps an error of a fewpercentage points in its results. They also introduce noise into theircomputations, so the computations are not repeatable. Further, theyrepresent only a small range of values, corresponding for instance to 8bit fixed point values rather than providing the large dynamic range oftypical 32 or 64 bit floating point representations. Given theselimitations, the SCAMP was not intended as a general purpose computer,but instead was designed and used for image processing and for modelingbiological early vision processes. Such applications do not require afull range of arithmetic operations in hardware, and the SCAMP, forexample, omits general division and multiplication from its design.

While SIMD machines were popular in the 1980s, as price/performance formicroprocessors improved designers began building machines from largecollections of communicating microprocessors. These MIMD machines arefast and can have price/performance comparable to their componentmicroprocessors, but they exhibit the same inefficiency as thosecomponents in that they deliver to their software relatively littlecomputation per transistor.

Field Programmable Gate Arrays (FPGAs) are integrated circuitscontaining a large grid of general purpose digital elements withreconfigurable wiring between those elements. The elements originallywere single digital gates, such as AND and OR gates, but evolved tolarger elements that could, for instance, be programmed to map 6 inputsto 1 output according to any Boolean function. This architecture allowsthe FPGA to be configured from external sources to perform a widevariety of digital computations, which allows the device to be used as aco-processor to a CPU to accelerate computation. However, arithmeticoperations such as multiplication and division on integers, andespecially on floating point numbers, require many gates and can absorba large fraction of an FPGA's general purpose resources. For thisreason, modern FPGAs often devote a significant portion of their area toproviding dozens or hundreds of multiplier blocks, which can be usedinstead of general purpose resources for computations requiringmultiplication. These multiplier blocks typically perform 18 bit orwider integer multiplies, and use many transistors, as similarmultiplier circuits do when they are part of a general purpose CPU.

Existing Field Programmable Analog Arrays (FPAAs) are analogous toFPGAs, but their configurable elements perform analog processing. Thesedevices generally are intended to do signal processing, such as helpingmodel neural circuitry. They are relatively low precision, haverelatively low dynamic range, and introduce noise into computation. Theyhave not been designed as, or intended for use as, general purposecomputers. For instance, they are not seen by those having ordinaryskill in the art as machines that can run the variety of complexalgorithms with floating point arithmetic that typically run on highperformance digital computers.

Finally, Graphics Processing Units (GPUs) are a variety of parallelprocessor that evolved to provide high speed graphics capabilities topersonal computers. They offer standard floating point computingabilities with very high performance for certain tasks. Their computingmodel is sometimes based on having thousands of nearly identical threadsof computing (SIMT), which are executed by a collection of SIMD-likeinternal computing engines, each of which is directed and redirected toperform work for which a slow external DRAM memory has provided data.Like other machines that implement standard floating point arithmetic,they use many transistors for that arithmetic. They are as wasteful ofthose transistors, in the sense discussed above, as are general purposeCPUs.

Some GPUs include support for 16 bit floating point values (sometimescalled the “Half” format). The GPU manufacturers, currently such asNVIDIA or AMD/ATI, describe this capability as being useful forrendering images with higher dynamic range than the usual 32 bit RGBAformat, which uses 8 bits of fixed point data per color, while alsosaving space over using 32 bit floating point for color components. Thespecial effects movie firm Industrial Light and Magic (ILM)independently defined an identical representation in their OpenEXRstandard, which they describe as “a high dynamic-range (HDR) image fileformat developed by Industrial Light & Magic for use in computer imagingapplications.” Wikipedia (late 2008) describes the 16 bit floating pointrepresentation thusly: “This format is used in several computer graphicsenvironments including OpenEXR, OpenGL, and D3DX. The advantage over8-bit or 16-bit binary integers is that the increased dynamic rangeallows for more detail to be preserved in highlights and shadows. Theadvantage over 32-bit single precision binary formats is that itrequires half the storage and bandwidth.”

When a graphics processor includes support for 16 bit floating point,that support is alongside support for 32 bit floating point, andincreasingly, 64 bit floating point. That is, the 16 bit floating pointformat is supported for those applications that want it, but the higherprecision formats also are supported because they are believed to beneeded for traditional graphics applications and also for so called“general purpose” GPU applications. Thus, existing GPUs devotesubstantial resources to 32 (and increasingly 64) bit arithmetic and arewasteful of transistors in the sense discussed above.

The variety of architectures mentioned above are all attempts to getmore performance from silicon than is available in a traditionalprocessor design. But designers of traditional processors also have beenstruggling to use the enormous increase in available transistors toimprove performance of their machines. These machines often arerequired, because of history and economics, to support large existinginstruction sets, such as the Intel x86 instruction set. This isdifficult, because of the law of diminishing returns, which does notenable twice the performance to be delivered by twice the transistorcount. One facet of these designers' struggle has been to increase theprecision of arithmetic operations, since transistors are abundant andsome applications could be sped up significantly if the processornatively supported long (e.g., 64 bit) numbers. With the increase ofnative fixed point precision from 8 to 16 to 32 to 64 bits, and offloating point from 32 to 64 and sometimes 128 bits, programmers havecome to think in terms of high precision and to develop algorithms basedon the assumption that computer processors provide such precision, sinceit comes as an integral part of each new generation of silicon chips andthus is “free.”

Embodiments of the present invention efficiently provide computing powerusing a fundamentally different approach than those described above. Inparticular, embodiments of the present invention are directed tocomputer processors or other devices which use low precision highdynamic range (LPHDR) processing elements to perform computations (suchas arithmetic operations).

One variety of LPHDR arithmetic represents values from one millionth upto one million with a precision of about 0.1%. If these values wererepresented and manipulated using the methods of floating pointarithmetic, they would have binary mantissas of no more than 10 bitsplus a sign bit and binary exponents of at least 5 bits plus a sign bit.However, the circuits to multiply and divide these floating point valueswould be relatively large. One example of an alternative embodiment isto use a logarithmic representation of the values. In such an approach,the values require the same number of bits to represent, butmultiplication and division are implemented as addition and subtraction,respectively, of the logarithmic representations. Addition andsubtraction may be implemented efficiently as described below. As aresult, the area of the arithmetic circuits remains relatively small anda greater number of computing elements can be fit into a given area ofsilicon. This means the machine can perform a greater number ofoperations per unit of time or per unit power, which gives it anadvantage for those computations able to be expressed in the LPHDRframework.

Another embodiment is to use analog representations and processingmechanisms. Analog implementation of LPHDR arithmetic has the potentialto be superior to digital implementation, because it tends to use thenatural analog physics of transistors or other physical devices insteadof using only the digital subset of the device's behavior. This fulleruse of the devices' natural abilities may permit smaller mechanisms fordoing LPHDR arithmetic. In recent years, in the field of siliconcircuitry, analog methods have been supplanted by digital methods. Inpart, this is because of the ease of doing digital design compared toanalog design. Also in part, it is because of the continued rapidscaling of digital technology (“Moore's Law”) compared to analogtechnology. In particular, at deep submicron dimensions, analogtransistors no longer work as they had in prior generations oflarger-scale technology. This change of familiar behavior has madeanalog design still harder in recent years. However, digital transistorsare in fact analog transistors used in a digital way, meaning digitalcircuits are really analog circuits designed to attempt to switch thetransistors between completely on and completely off states. As scalingcontinues, even this use of transistors is starting to come face to facewith the realities of analog behavior. Scaling of transistors fordigital use is expected either to stall or to require digital designersincreasingly to acknowledge and work with analog issues. For thesereasons, digital embodiments may no longer be easy, reliable, andscalable, and analog embodiments of LPHDR arithmetic may come todominate commercial architectures.

Because LPHDR processing elements are relatively small, a singleprocessor or other device may include a very large number of LPHDRprocessing elements, adapted to operate in parallel with each other, andtherefore may constitute a massively parallel LPHDR processor or otherdevice. Such a processor or other device has not been described orpracticed as a means of doing general purpose computing by those havingordinary skill in the art for at least two reasons. First, it iscommonly believed by those having ordinary skill in the art, that LPHDRcomputation, and in particular massive amounts of LPHDR computation,whether performed in a massively parallel way or not, is not practicalas a substrate for moderately general computing. Second, it is commonlybelieved by those having ordinary skill in the art that massive amountsof even high precision computation on a single chip or in a singlemachine, as is enabled by a compact arithmetic processing unit, is notuseful without a corresponding increase in bandwidth between processingelements within the machine and into and out of the machine becausecomputing is wire limited and arithmetic can be considered to beavailable at no cost.

Despite these views—that massive amounts of arithmetic on a chip or in amassively parallel machine are not useful, and that massive amounts ofLPHDR arithmetic are even worse—embodiments of the present inventiondisclosed herein demonstrate that massively parallel LPHDR designs arein fact useful and provide significant practical benefits in at leastseveral significant applications.

To conclude, modern digital computing systems provide high precisionarithmetic, but that precision is costly. A modern double precisionfloating point multiplier may require on the order of a milliontransistors, even though only a handful of transistors is required toperform a low precision multiplication. Despite the common belief amongthose having ordinary skill in the art that modern applications requirehigh precision processing, in fact a variety of useful algorithmsfunction adequately at much lower precision. As a result, suchalgorithms may be performed by processors or other devices implementedaccording to embodiments of the present invention, which come closer toachieving the goal of using a few transistors to multiply and a wirejunction to add, thus enabling massively parallel arithmetic computationto be performed with relatively small amounts of physical resources(such as a single silicon chip). Although certain specialized tasks canfunction at low precision, it is not obvious, and in fact has beenviewed as clearly false by those having ordinary skill in the art, thatrelatively general purpose computing such as is typically performedtoday on general purpose computers can be done at low precision.However, in fact a variety of useful and important algorithms can bemade to function adequately at much lower than 32 bit precision in amassively parallel computing framework, and certain embodiments of thepresent invention support such algorithms, thereby offering much moreefficient use of transistors, and thereby provide improved speed, power,and/or cost, compared to conventional computers.

Various computing devices implemented according to embodiments of thepresent invention will now be described. Some of these embodiments maybe an instance of a SIMD computer architecture. Other architectures maybe used, such as MIMD architectures, programmable array architectures(such as FPGAs and FPAAs), or GPU/SIMT architectures. The techniquesdisclosed herein may, for example, be implemented using any processor orother device having such an existing architecture, and replacing oraugmenting some or all existing arithmetic units in the processor orother device, if any, with LPHDR arithmetic units in any of the waysdisclosed herein. Devices implemented according to embodiments of thepresent invention, however, need not start with an existing processordesign, but instead may be designed from scratch to include LPHDRarithmetic units within any of the architectures just described, or anyother architecture.

Embodiments of the present invention may, for example, be implementedusing the architecture of a particular kind of SIMD computer, the arrayprocessor. There are many variations and specific instances of arrayprocessors described in the scientific and commercial literature.Examples include the Illiac 4, the Connection Machine 1 and 2, theGoodyear MPP, and the MasPar line of computers.

Embodiments of the present invention need not, however, be implementedas SIMD computers. For example, embodiments of the present invention maybe implemented as FPGAs, FPAAs, or related architectures that providefor flexible connectivity of a set of processing elements. For example,embodiments of the present invention may be implemented as GPU/SIMTs andas MIMDs, among others. For example, embodiments of the presentinvention may be implemented as any kind of machine which uses LPHDRarithmetic processing elements to provide computing using a small amountof resources (e.g., transistors or volume) compared with traditionalarchitectures. Furthermore, references herein to “processing elements”within embodiments of the present invention should be understood moregenerally as any kind of execution unit, whether for performing LPHDRoperations or otherwise.

An example SIMD computing system 100 is illustrated in FIG. 1. Thecomputing system 100 includes a collection of many processing elements(PEs) 104. Sometimes present are a control unit (CU) 106, an I/O unit(IOU) 108, various Peripheral devices 110, and a Host computer 102. Thecollection of PEs is referred to herein as “the Processing ElementArray” (PEA), even though it need not be two-dimensional or an array orgrid or other particular layout. Some machines include additionalcomponents, such as an additional memory system called the “StagingMemory” in the Goodyear MPP, but these additional elements are neitheressential in the computer nor needed to understand embodiments of thepresent invention and therefore are omitted here for clarity ofexplanation. One embodiment of the present invention is a SIMD computingsystem of the kind shown in FIG. 1, in which one or more (e.g., all) ofthe PEs in the PEA 104 are LPHDR elements, as that term is used herein.

The Host 102 is responsible for overall control of the computing system100. It performs the serial, or mostly serial, computation typical of atraditional uni-processor. The Host 102 could have more complicatedstructure, of course, including parallelism of various sorts. Indeed aheterogeneous computing system combining multiple computingarchitectures in a single machine is a good use for embodiments of thepresent invention.

A goal of the Host 102 is to have the PEA 104 perform massive amounts ofcomputation in a useful way. It does this by causing the PEs to performcomputations, typically on data stored locally in each PE, in parallelwith one another. If there are many PEs, much work gets done during eachunit of time.

The PEs in the PEA 104 may be able to perform their individualcomputations roughly as fast as the Host 102 performs its computations.This means it may be inefficient to have the Host 102 attempt to controlthe PEA 104 on a time scale as fine as the Host's or PEA's minimal timestep. (This minimal time, in a traditional digital design, would be theclock period.) For this reason, the specialized control unit (CU) 106may be included in the architecture. The CU 106 has the primary task ofretrieving and decoding instructions from an instruction memory, whichconceptually is part of the CU 106, and issuing the partially decodedinstructions to all the PEs in the PEA 104. (This may be viewed by theCU software as happening roughly simultaneously for all the PEs, thoughit need not literally be synchronous, and in fact it may be effective touse an asynchronous design in which multiple instructions at differentstages of completion simultaneously propagate gradually across the PEA,for instance as a series of wave fronts.)

In a design which includes the CU 106, the Host 102 typically will loadthe instructions (the program) for the PEA 104 into the CU instructionmemory (not shown in FIG. 1), then instruct the CU 106 to interpret theprogram and cause the PEA 104 to compute according to the instructions.The program may, for example, look generally similar to a typicalmachine language program, with instructions to cause data movement,logical operations, arithmetic operations, etc., in and between the PEsand other instructions to do similar operations together with controlflow operations within the CU 106. Thus, the CU 106 may run a typicalsort of program, but with the ability to issue massively parallelinstructions to the PEA 104.

In order to get data into and out of the CU 106 and PEA 104, the I/OUnit 108 may interface the CU 106 and PEA 104 with the Host 102, theHost's memory (not shown in FIG. 1), and the system's Peripherals 110,such as external storage (e.g., disk drives), display devices forvisualization of the computational results, and sometimes special highbandwidth input devices (e.g., vision sensors). The PEA's ability toprocess data far faster than the Host 102 makes it useful for the IOU108 to be able to completely bypass the Host 102 for some of its datatransfers. Also, the Host 102 may have its own ways of communicatingwith the Peripherals 110.

The particular embodiment illustrated in FIG. 1 is shown merely forpurposes of example and does not constitute a limitation of the presentinvention. For example, alternatively the functions performed by the CU106 could instead be performed by the Host 102 with the CU 106 omitted.The CU 106 could be implemented as hardware distant from the PEA 104(e.g., off-chip), or the CU 106 could be near to the PEA 104 (e.g.,on-chip). I/O could be routed through the CU 106 with the IOU 108omitted or through the separate I/O controller 108, as shown.Furthermore, the Host 102 is optional; the CU 106 may include, forexample, a CPU, or otherwise include components sufficient to replacethe functions performed by the Host 102. The Peripherals 110 shown inFIG. 1 are optional. The design shown in FIG. 1 could have a specialmemory, such as the Goodyear MPP's “staging memory,” which provides anintermediate level of local storage. Such memory could, for example, bebonded to the LPHDR chip using 3D fabrication technology to providerelatively fast parallel access to the memory from the PEs in the PEA104.

The PEA 104 itself, besides communicating with the CU 106 and IOU 108and possibly other mechanisms, has ways for data to move within thearray. For example, the PEA 104 may be implemented such that data maymove from PEs only to their nearest neighbors, that is, there are nolong distance transfers. FIGS. 2 and 3 show embodiments of the presentinvention which use this approach, where the nearest neighbors are thefour adjacent PEs toward the North, East, West, and South, called a NEWSdesign. For example, FIG. 2 shows a subset of the PEs in PEA 104, namelyPE 202, PE 204, PE 206, PE 208, and PE 210. When the CU 106 issues datamovement instructions, all the PEs access data from or send data totheir respective specified nearest neighbor. For instance, every PEmight access a specified data value in its neighbor to the West and copyit into its own local storage. In some embodiments, such as some analogembodiments, these kinds of transfers may result in some degradation ofthe value copied.

FIG. 3 shows a PE 302 that includes data connections to the IOU 108. PE302 is connected at the North to PE 304, at the East to PE 306, at theSouth to PE 308, and at the West to PE 310. However, driving signalsfrom inside the PEA 104 out to the IOU 108 usually requires a physicallyrelatively large driving circuit or analogous mechanism. Having those atevery PE may absorb much of the available resources of the hardwareimplementation technology (such as VLSI area). In addition, havingindependent connections from every PE to the IOU 108 means many suchconnections, and long connections, which also may absorb much of theavailable hardware resources. For these reasons, the connections betweenthe PEs and the IOU 108 may be limited to those PEs at the edges of thePE array 104. In this case, to get data out of, and perhaps into, thePEA 104, the data is read and written at the edges of the array and CUinstructions are performed to shift data between the edges and interiorof the PEA 104. The design may permit data to be pushed from the IOU 108inward to any PE in the array using direct connections, but may requirereadout to occur by using the CU 106 to shift data to the edges where itcan be read by the IOU 108.

Connections between the CU 106 and PEA 104 have analogous variations.One design may include the ability to drive instructions into all thePEs roughly simultaneously, but another approach is to have theinstructions flow gradually (for instance, shift in discrete time steps)across the PEA 104 to reach the PEs. Some SIMD designs, which may beimplemented in embodiments of the present invention, have a facility bywhich a “wired-or” or “wired- and” of the state of every PE in the PEA104 can be read by the CU 106 in approximately one instruction delaytime.

There are many well studied variations on these matters in theliterature, any of which may be incorporated into embodiments of thepresent invention. For example, an interconnect, such as an 8-way localinterconnect, may be used. The local connections may include a mixtureof various distance hops, such as distance 4 or 16 as well as distance1. The outside edges may be connected using any topology, such as atorus or twisted torus. Instead of or in addition to a localinterconnect, a more complex global interconnect, such as the hypercubedesign, may be used. Furthermore, the physical implementation of the PEA104 (e.g., a chip) could be replicated (e.g., tiled on a circuit board)to produce a larger PEA. The replication may form a simple grid or otherarrangement, just as the component PEAs may but need not be grids.

FIG. 4 shows an example design for a PE 400 (which may be used toimplement any one or more of the PEs in the PEA 104). The PE 400 storeslocal data. The amount of memory for the local data varies significantlyfrom design to design. It may depend on the implementation technologiesavailable for fabricating the PE 400. Sometimes rarely changing values(Constants) take less room than frequently changing values (Registers),and a design may provide more Constants than Registers. For instance,this may be the case with digital embodiments that use single transistorcells for the Constants (e.g., floating gate Flash memory cells) andmultiple transistor cells for the Registers (e.g., 6-transistor SRAMcells). Sometimes the situation is reversed, as may be the case inanalog embodiments, where substantial area for capacitance may be neededto ensure stable long term storage of Constants, and such embodimentsmay have more Registers than Constants. Typical storage capacities mightbe tens or hundreds of arithmetic values stored in the Registers andConstants in each PE, but these capacities are adjustable by thedesigner. Some designs, for instance, may have Register storage but noConstant storage. Some designs may have thousands or even many morevalues stored in each PE. All of these variations may be reflected inembodiments of the present invention.

Each PE needs to operate on its local data. For this reason within thePE 400 there are data paths 402 a-i, routing mechanisms (such as themultiplexor MUX 404), and components to perform some collection oflogical and arithmetic operations (such as the logic unit 406 and theLPHDR arithmetic unit 408). The LPHDR arithmetic unit 408 performs LPHDRarithmetic operations, as that term is used herein. The input, output,and intermediate “values” received by, output by, and operated on by thePE 400 may, for example, take the form of electrical signalsrepresenting numerical values.

The PE 400 also may have one or more flag bits, shown as Mask 410 inFIG. 4. The purpose of the Mask 410 is to enable some PEs, the ones inwhich a specified Mask bit is set, to ignore some instructions issued bythe CU 106. This allows some variation in the usual lock-step behaviorsof all PEs in the PEA 104. For instance, the CU 106 may issue aninstruction that causes each PE to reset or set its Mask 410 dependingon whether a specified Register in the PE is positive or negative. Asubsequent instruction, for instance an arithmetic instruction, mayinclude a bit meaning that the instruction should be performed only bythose PEs whose Mask 410 is reset. This combination has the effect ofconditionally performing the arithmetic instruction in each PE dependingon whether the specified Register in that PE was positive. As with theCompare instructions of traditional computers, there are many possibledesign choices for mechanisms to set and clear Masks.

The operation of the PEs is controlled by control signals 412 a-dreceived from the CU 106, four of which are shown in FIG. 4 merely forpurposes of example and not limitation. We have not shown details ofthis mechanism, but the control signals 412 a-d specify which Registeror Constant memory values in the PE 400 or one of its neighbors to sendto the data paths, which operations should be performed by the Logic 406or Arithmetic 408 or other processing mechanisms, where the resultsshould be stored in the Registers, how to set, reset, and use the Mask410, and so on. These matters are well described in the literature onSIMD processors.

Many variations of this PE 400 and PEA design are possible and fallwithin the scope of the present invention. Digital PEs can haveshifters, lookup tables, and many other mechanisms such as described inthe literature. Analog PEs can have time-based operators, filters,comparators with global broadcast signals and many other mechanisms suchas described in the literature. The PEA 104 can include globalmechanisms such as wired-OR or wired-AND for digital PEAs or wired-SUMfor analog PEAs Again, there are many variations well described in theliterature on digital and analog computing architectures.

For example, LPHDR operations other than and/or in addition to additionand multiplication may be supported. For example, a machine which canonly perform multiplication and the function (1-X) may be used toapproximate addition and other arithmetic operations. Other collectionsof LPHDR operations may be used to approximate LPHDR arithmeticoperations, such as addition, multiplication, subtraction, and division,using techniques that are well-known to those having ordinary skill inthe art.

One aspect of embodiments of the present invention that is unique is theinclusion of LPHDR arithmetic mechanisms in the PEs. Embodiments of suchmechanisms will now be described.

One digital embodiment of the LPHDR arithmetic unit 408 operates ondigital (binary) representations of numbers. In one digital embodimentthese numbers are represented by their logarithms. Such a representationis called a Logarithmic Number System (LNS), which is well-understood bythose having ordinary skill in the art.

In an LNS, numbers are represented as a sign and an exponent. There isan implicit base for the logarithms, typically 2 when working withdigital hardware. In the present embodiment, a base of 2 is used forpurposes of example. As a result, a value, say B, is represented by itssign and a base 2 logarithm, say b, of its absolute value. For numbersto have representation errors of at most, say, 1% (one percent), thefractional part of this logarithm should be represented with enoughprecision that the least possible change in the fraction corresponds toabout a 1% change in the value B. If fractions are represented using 6bits, increasing or decreasing the fraction by 1 corresponds tomultiplying or dividing B by the 64th root of 2, which is approximately1.011. This means that numbers may be represented in the presentembodiment with a multiplicative error of approximately 1%. So, in thisexample embodiment the fraction part of the representation has 6 bits.

Furthermore, the space of values processed in the present embodimenthave high dynamic range. To represent numbers whose absolute value isfrom, say, one billionth to one billion, the integer part of thelogarithm must be long enough to represent plus or minus the base 2logarithm of one billion. That logarithm is about 29.9. In the presentembodiment the integer part of the logarithm representation is 5 bitslong to represent values from 0 to 31, which is sufficient. There alsois a sign bit in the exponent. Negative logarithms are represented usingtwo's complement representation.

In an LNS, the value zero corresponds to the logarithm negativeinfinity. One can choose a representation to explicitly represent thisspecial value. However, to minimize resources (for instance, area) usedby arithmetic circuits, the present embodiment represents zero by themost negative possible logarithm, which is −32, corresponding to thetwo's complement bit representation ‘100000 000000’, and denoting avalue of approximately 2.33E-10.

When computing, situations can arise in which operations cannot producereasonable values. An example is when a number is too large to berepresented in the chosen word format, such as when multiplying oradding two large numbers or upon divide by zero (or nearly zero). Onecommon approach to this problem is to allow a value to be marked as NotA Number (NAN) and to make sure that each operation produces NAN if aproblem arises or if either of its inputs is NAN. The present embodimentuses this approach, as will be described in the following.

FIG. 5 shows the word format 500 for these numbers, in the presentembodiment. It has one NAN bit 502 a, one bit 502 b for the sign of thevalue, and 12 bits 502 c-e representing the logarithm. The logarithmbits include a 5 bit integer part 502 d and a 6 bit fraction part 502 e.To permit the logarithms to be negative, there is a sign bit 502 c forthe logarithm which is represented in two's complement form. The NAN bitis set if some problem has arisen in computing the value. The wordformat 500 shown in FIG. 5 is merely an example and does not constitutea limitation of the present invention. Other variations may be used, solong as they have low precision and high dynamic range.

FIG. 6 shows an example digital implementation of the LPHDR arithmeticunit 408 for the representation illustrated in FIG. 5. The unit 408receives two inputs, A 602 a and B 602 b, and produces an output 602 c.The inputs 602 a-b and output 602 c may, for example, take the form ofelectrical signals representing numerical values according to therepresentation illustrated in FIG. 5, as is also true of signalstransmitted within the unit 408 by components of the unit 408. Theinputs 602 a-b and output 602 c each are composed of a Value and a NAN(Not A Number) bit. The unit 408 is controlled by control signals 412a-d, coming from the CU 106, that determine which available arithmeticoperation will be performed on the inputs 602 a-b. In this embodiment,all the available arithmetic operations are performed in parallel on theinputs 602 a-b by adder/subtractor 604, multiplier 606, and divider 608.Adder/subtractor 604 performs LPHDR addition and subtraction, multiplier606 performs LPHDR multiplication, and divider 608 performs LPHDRdivision.

The desired result (from among the outputs of adder/subtractor 604,multiplier 606, and divider 608) is chosen by the multiplexers (MUXes)610 a and 610 b. The right hand MUX 610 b sends the desired value to theoutput 602 c. The left hand MUX 610 a sends the corresponding NAN bitfrom the desired operation to the OR gate 612, which outputs a set NANbit if either input is NAN or if the specified arithmetic operationyields NAN. The computing architecture literature discusses manyvariations which may be incorporated into the embodiment illustrated inFIG. 6.

LNS arithmetic has the great advantage that multiplication (MUL) anddivision (DIV) are very easy to compute and take few physical resources(e.g., little area in a silicon implementation). The sign of the resultis the exclusive-or of the signs of the operands. The logarithm part ofthe output is the sum, in the case of MUL, or the difference, in thecase of DIV, of the logarithm parts of the operands. The sum ordifference of the logarithms can overflow, producing a NAN result.Certain other operations also are easy in LNS arithmetic. For instance,square root corresponds to dividing the logarithm in half, which in ourrepresentation means simply shifting it one bit position to the right.

Thus, the multiplier 606 and divider 608 in FIG. 6 are implemented ascircuits that simply add or subtract their inputs, which are two'scomplement binary numbers (which in turn happen to be logarithms). Ifthere is overflow, they output a 1 for NAN.

Implementing addition and subtraction in LNS, that is, theadder/subtractor 604 in FIG. 6, follows a common approach used in theliterature on LNS. Consider addition. If we have two positive numbers Band C represented by their logarithms b and c, the representation of thesum of B and C is log(B+C). An approach to computing this result that iswell known to those skilled in the art is based on noticing thatlog(B+C)=log(B*(1+C/B))=log(B)+log(1+C/B)=b+F(c−b) where F(x)=log(1+2̂x).Thus, the present embodiment computes c−b, feeds that through F, andadds the result to b, using standard digital techniques known to thoseskilled in the art.

Much of the published literature about LNS is concerned with how tocompute F(x), the special function for ADD, along with a similarfunction for SUB. Often these two functions share circuitry, and this iswhy a single combined adder/subtractor 604 is used in the embodiment ofFIG. 6. There are many published ways to compute these functions orapproximations to them, including discussions of how to do this when thevalues are of low precision. Any such method, or other method, may beused. Generally speaking, the more appropriate variations for massivelyparallel LPHDR arithmetic are those that require the minimal use ofresources, such as circuit area, taking advantage of the fact that therepresentation used in the embodiment of FIG. 6 is low precision andthat the arithmetic operations need not be deterministic nor return themost accurate possible answer within the low precision representation.Thus, embodiments of the present invention may use circuitry that doesnot compute the best possible answer, even among the limited choicesavailable in a low precision representation.

In order to enable conditional operation of selected PEs, the presentembodiment is able to reset and set the MASK flag 410 based on resultsof computations. The mechanism for doing this is that the CU 106includes instructions that cause the MASK 410 in each PE tounconditionally reset or set its flag along with other instructions toperform basic tests on values entering the MASK 410 on data path 402 fand to set the flag accordingly. Examples of these latter instructionsinclude copying the sign bit or NAN bit of the word on data path 402 finto the MASK bit 410. Another example is to set the MASK bit 410 if the12 bit value part of the word on data path 402 f is equal to binaryzero. There are many additional and alternative ways for doing this thatare directly analogous to comparison instructions in traditionalprocessors and which are well understood by those skilled in the art.

It is worth noting that while the obvious method of using the above LNSoperations is to do LPHDR arithmetic, the programmer also may considerselected values to be 12 bit two's complement binary numbers. MUL andDIV may be used to add and subtract such values, since that is preciselytheir behavior in LNS implementations. The Mask setting instructions cancompare these simple binary values. So besides doing LPHDR computations,this digital embodiment using LNS can perform simple binary arithmeticon short signed integers.

Some embodiments of the present invention may include analogrepresentations and processing methods. Such embodiments may, forexample, represent LPHDR values as charges, currents, voltages,frequencies, pulse widths, pulse densities, various forms of spikes, orin other forms not characteristic of traditional digitalimplementations. There are many such representations discussed in theliterature, along with mechanisms for processing values so represented.Such methods, often called Analog methods, can be used to perform LPHDRarithmetic in the broad range of architectures we have discussed, ofwhich SIMD is one example.

An example of an analog SIMD architecture is the SCAMP design (andrelated designs) of Dudek. In that design values have low dynamic range,being accurate roughly to within 1%. Values are represented by chargeson capacitors. Those capacitors typically are the gates of transistors.Each PE has several memory cells, analogous to the Registers shown inFIG. 4. Addition is performed by turning on pass transistors from thetwo operands, which transfer their charge onto an analog bus, where itis summed by the natural physics of charge and wires, upon which it isgated onto another Register to charge up its capacitor, which thenrepresents the sum of the operands. The detailed mechanism disclosed byDudek actually produces the negative of the sum, but the basic conceptis as described and is a simple way to perform addition and subtractionusing analog representations and simple processing mechanisms.

Variations of the SCAMP design have been fabricated and used to performa range of low precision, low dynamic range computations related toimage processing. These designs do not perform high dynamic rangearithmetic, nor do they include mechanisms for performing multiplicationor division of values stored in Registers. However, the Dudek designssuggest the general feasibility of constructing analog SIMD machines.The following describes how to build an analog SIMD machine thatperforms LPHDR arithmetic, and is thus an embodiment of the presentinvention.

One embodiment of the present invention represents values as a mixtureof analog and digital forms. This embodiment represents values as lowprecision, normalized, base 2 floating point numbers, where the mantissais an analog value and the exponent is a binary digital value. Theanalog value may be accurate to about 1%, following the approach ofDudek, which is well within the range of reasonable analog processingtechniques. The exponent may be 6 bits long, or whatever is needed toprovide the desired high dynamic range.

To multiply values, the embodiment proceeds by analogy to traditionalfloating point methods. The digital exponents are summed using a binaryarithmetic adder, a standard digital technique. The analog mantissas aremultiplied. Since they represent normalized values between approximately½ and 1, their product may be as small as approximately ¼. Such aproduct value needs to be normalized back to the range ½ to 1. This isdone, in the present embodiment, by comparing the analog mantissa to ananalog representation of ½, using a threshold circuit. If the mantissais below ½, then it is doubled and one is subtracted from the exponent,where such subtraction is simple digital subtraction. Doubling themantissa is implemented in a way that corresponds to the chosen analogrepresentation. For example, whatever means are being used to add twoanalog values can be used to double the mantissa, by adding it to a copyof itself. For example, if the mantissa is represented as a current,such as copy may be produced by a current mirror, or other suitablemechanism, and addition may be performed by a current summing junction.

The means of multiplying the original analog mantissas depends on therepresentation chosen. For example, if mantissas are represented usingcharge, following SCAMP, then any known method from the literature maybe used to convert charge to current. For instance, since the charge ona capacitor determines the voltage on the capacitor, this may beimplemented as a conversion from voltage to current, which is a basictechnique in analog electronics known to those skilled in the art. Inany case, if the mantissas are represented as currents, or once themantissas are converted to currents, they may be multiplied using, forinstance, the techniques of Gilbert. The Gilbert multiplier produces acurrent, representing the product, which may, if necessary, then beconverted back to charge (or whatever representation is used). These aremerely examples of how the needed operations might be performed. Theliterature discusses these matters extensively and these kinds of analogcircuits are known to those skilled in the art.

Adding and subtracting values requires pre-normalization of the valuesto the same exponent, as is done in traditional digital floating pointarithmetic. The present embodiment does this by comparing the exponentsand choosing the smaller one. Then the smaller one is subtracted fromthe larger, using digital means. The difference specifies how many timesthe mantissa which corresponds to the smaller exponent needs to bedivided in half. If that mantissa is represented by (or converted to) acurrent, then an analog R-2R style ladder may be used to divide thecurrent in half the required number of times, with the stage of theladder specified by the difference of exponents calculated as above. Theresulting scaled down current is added to (or subtracted from, if thisis an LPHDR subtraction operation) the current corresponding to themantissa associated with the larger exponent to yield the outputmantissa. The output exponent associated with the output mantissa is thelarger exponent. Post-normalization may be needed at this point. If theoutput mantissa is greater than 1, then it needs to be divided in halfand the output exponent needs to be incremented. If it is less than ½,then it needs to be doubled enough times to exceed ½ and the outputexponent must be decremented correspondingly, which may be performed bya series of threshold circuits, doubler circuits, and associateddecrementer circuits. These increments and decrements of the binarydigital exponent, and corresponding doublings and halvings of the analogmantissa current, are straightforward operations well known to thoseskilled in the art.

The present embodiment represents the exponent as a digital binarynumber. Alternate embodiments may represent the exponent as an analogvalue. However, it is important that the exponent be represented instorage and computation in such a manner that neither noise nor othererrors cause a change in the value it represents. Such changes in theexponent could introduce factors of two (or in some embodiments larger)changes in the values of the stored numbers. To maintain accuracy of theexponents, an embodiment may quantize the exponent to relatively fewlevels, for instance 16 values plus a sign bit. During processing,slight variations in the analog representation of the exponent may thenbe removed by circuitry that restores values to the 16 standardquantization levels. To get sufficient dynamic range in such anembodiment, the floating point numbers may be processed as base 4numbers, rather than the usual base 2 numbers. This means, for instance,that normalized mantissas are in the range ¼ to 1. The methods discussedabove for addition, subtraction, and multiplication apply as described,with slight and straightforward variations.

The analog and mixed signal embodiments discussed above are merelyexamples and do not constitute a limitation of the present invention.The published literature on neuromorphic, analog, and mixed signaltechniques provides a wealth of methods that enable LPHDR storage andprocessing to be implemented. Such storage and processing may introducenoise as well as fabrication errors into the behavior of machinesperforming LPHDR arithmetic. The results we present below, on softwareapplications running using “fp+noise” arithmetic, show that despitethese very “un-digital” qualities a machine built in this way issurprisingly useful.

Evidence that LPHDR arithmetic is useful in several important practicalcomputing applications will now be provided. The evidence is presentedfor a broad variety of embodiments of the present invention, therebyshowing that the usefulness does not depend much on the detailedimplementation.

For the goal of showing usefulness, we choose a very general embodimentof an LPHDR machine. Our model of the machine is that it provides atleast the following capabilities: (1) is massively parallel, (2)provides LPHDR arithmetic possibly with noise, (3) provides a smallamount of memory local to each arithmetic unit, (4) provides thearithmetic/memory units in a two-dimensional physical layout with onlylocal connections between units (rather than some more powerful,flexible, or sophisticated connection mechanism), and (5) provides onlylimited bandwidth between the machine and the host machine. Note thatthis model is merely an example which is used for the purpose ofdemonstrating the utility of various embodiments of the presentinvention, and does not constitute a limitation of the presentinvention. This model includes, among others, implementations that aredigital or analog or mixed, have zero or more noise, have architectureswhich are FPGA-like, or SIMD-like, or MIMD-like, or otherwise meet theassumptions of the model. More general architectures, such as sharedmemory designs, GPU-like designs, or other sophisticated designs subsumethis model's capabilities, and so LPHDR arithmetic in thosearchitectures also is useful. While we are thus showing that LPHDRarithmetic is useful for a broad range of designs, of which SIMD is onlyan instance, for purpose of discussion below, we call each unit, whichpairs memory with arithmetic, a Processing Element or “PE”.

Several applications are discussed below. For each, the discussion shows(1) that the results are useful when computation is performed inpossibly noisy LPHDR arithmetic, and (2) that the computation can bephysically laid out in two dimensions with only local flow of databetween units, only limited memory within each unit, and only limiteddata flow to/from the host machine, in such a way that the computationmakes efficient use of the machine's resources (area, time, power). Thefirst requirement is referred to as “Accuracy” and the secondrequirement “Efficiency.” Applications that meet both requirementsrunning in this model will function well on many kinds of LPHDRmachines, and thus those machines are a broadly useful invention.

Applications are tested using two embodiments for the machine'sarithmetic. One uses accurate floating point arithmetic but multipliesthe result of each arithmetic operation by a uniformly chosen randomnumber between 0.99 and 1.01. In the following discussion, thisembodiment is denoted “fp+noise”. It may represent the results producedby an analog embodiment of the machine.

A second embodiment uses logarithmic arithmetic with a valuerepresentation as shown in FIG. 5. The arithmetic is repeatable, thatis, not noisy, but because of the short fraction size it produces errorsof up to approximately 1-2% in each operation. In the followingdiscussion, this embodiment is denoted “Ins”. It may represent theresults produced by a particular digital embodiment of the machine.

To demonstrate usefulness of embodiments of the invention, we shalldiscuss three computational tasks that are enabled by embodiments of theinvention and which in turn enable a variety of practical applications.Two of the tasks are related to finding nearest neighbors and the otheris related to processing visual information. We shall describe thetasks, note their practical application, and then demonstrate that eachtask is solvable using the general model described above and thussolvable using embodiments of the present invention.

Application 1 Finding Nearest Neighbors

Given a large set of vectors, called Examples, and a given vector,called Test, the nearest neighbor problem (“NN”) is to find the Examplewhich is closest to Test where the distance metric is the square of theEuclidean distance (sum of squares of distances between respectivecomponents).

NN is a widely useful computation. One use is for data compression,where it is called “vector quantization”. In this application we have aset of relatively long vectors in a “code book” (these are the Examples)and associated short code words (for instance the index of the vector inthe code book). We move through a sequence of vectors to be compressed,and for each such vector (Test), find the nearest vector in the codebook and output the corresponding code word. This reduces the sequenceof vectors to the shorter sequence of code words. Because the code wordsdo not completely specify the original sequence of vectors, this is alossy form of data compression. Among other applications, it may be usedin speech compression and in the MPEG standards.

Another application of NN would be in determining whether snippets ofvideo occur in a large video database. Here we might abstract frames ofvideo from the snippet into feature vectors, using known methods, suchas color histograms, scale invariant feature extraction, etc. TheExamples would be analogous feature vectors extracted from the videodatabase. We would like to know whether any vector from the snippet wasclose to any vector from the database, which NN can help us decide.

In many applications of nearest neighbor, we would prefer to find thetrue nearest neighbor but it is acceptable if we sometimes find anotherneighbor that is only slightly farther away or if we almost always findthe true nearest neighbor. Thus, an approximate solution to the nearestneighbor problem is useful, especially if it can be computed especiallyquickly, or at low power, or with some other advantage compared to anexact solution.

We shall now show that approximate nearest neighbor is computable usingembodiments of the present invention in a way that meets the criteria ofAccuracy and Efficiency.

Algorithm.

The following describes an algorithm which may be performed by machinesimplemented according to embodiments of the present invention, such asby executing software including instructions for performing thealgorithm. The inputs to the algorithm are a set of Examples and a Testvector. The algorithm seeks to find the nearest (or almost nearest)Example to the Test.

In the simplest version of the algorithm, the number of Examples may beno larger than the number of PEs and each vector must be short enough tofit within a single PE's memory. The Examples are placed into thememories associated with the PEs, so that one Example is placed in eachPE. Given a Test, the Test is passed through all the PEs, in turn.Accompanying the Test as it passes through the PEs is the distance fromthe Test to the nearest Example found so far, along with informationthat indicates what PE (and thus what Example) yielded that nearestExample found so far. Each PE computes the distance between the Test andthe Example stored in that PE's memory, and then passes along the Testtogether with either the distance and indicator that was passed intothis PE (if the distance computed by this PE exceeded the distancepassed into the PE) or the distance this PE computed along withinformation indicating this PE's Example is the nearest so far (if thedistance computed by this PE is less than the distance passed into thePE). Thus, the algorithm is doing a simple minimization operation as theTest is passed through the set of PEs. When the Test and associatedinformation leave the last PE, the output is a representation of whichPE (and Example) was closest to the Test, along with the distancebetween that Example and the Test.

In a more efficient variant of this algorithm, the Test is first passedalong, for example, the top row, then every column passes the Test andassociated information downward, effectively doing a search in parallelwith other columns, and once the information reaches the bottom itpasses across the bottom row computing a minimum distance Example of allthe columns processed so far as it passes across the row. This meansthat the time required to process the Test is proportional to (thegreater of) the number of PEs in a row or column.

An enhancement of this algorithm proceeds as above but computes andpasses along information indicating both the nearest and the secondnearest Example found so far. When this information exits the array ofPEs, the digital processor that is hosting the PE array computes (inhigh precision) the distance between the Test and the two Examplesindicated by the PE array, and the nearer of the two is output as thelikely nearest neighbor to the Test.

Accuracy.

We expressed the arithmetic performed by the enhanced algorithmdescribed above as code in the C programming language. That codecomputes both nearest neighbors, which are discussed here, along withweighted scores, which are discussed below.

The C code performs the same set of arithmetic operations in the sameorder using the same methods of performing arithmetic as an actualimplementation of the present invention, such as one implemented inhardware. It thus yields the same results as the enhanced algorithmwould yield when running on an implementation of the present invention.(How the algorithm is organized to run efficiently on such animplementation is discussed below in the section on Efficiency.)

In particular, when computing the distance between the Test and eachExample, the code uses Kahan's method, discussed below, to perform thepossibly long summation required to form the sum of the squares of thedistances between vector components of the Test and Example.

The C code contains several implementations for arithmetic, as discussedabove. When compiled with “#define fp” the arithmetic is done using IEEEstandard floating point. If a command line argument is passed in toenable noisy arithmetic, then random noise is added to the result ofevery calculation. This is the “fp+noise” form of arithmetic. Whencompiled without “#define fp” the arithmetic is done using low precisionlogarithmic arithmetic with a 6 bit base-2 fraction. This is the “Ins”form of arithmetic.

When the code was run it produced traces showing the results of thecomputations it performed. These traces, shown below, show that withcertain command line arguments the enhanced algorithm yielded certainresults for LPHDR nearest neighbor calculations. These results providedetails showing the usefulness of this approach. We shall discuss theresults briefly here.

The first results are for “fp+noise”. Ten distinct runs were performed.Each run generated one million random Example vectors of length five,where each component of each vector was drawn from N(0,1)—the Gaussian(normal) distribution with mean zero and standard deviation 1. Each runthen generated one hundred Test vectors of length five, where eachcomponent of each vector also was drawn from N(0,1). For each Test, thenearest neighbor was computed both according to the enhanced algorithmabove and according to the standard nearest neighbor method using highprecision floating point arithmetic. A count was kept of the number oftimes the enhanced algorithm yielded the same result as the standardfloating point method. The results were as follows:

-   -   % ./a.out 5 10 1000000 100 1    -   Representation is Floating Point with noise.    -   Run 1. On 100 tests, 100(100.0%) matches and 0.81% mean score        error.    -   Run 2. On 100 tests, 100(100.0%) matches and 0.84% mean score        error.    -   Run 3. On 100 tests, 100(100.0%) matches and 0.98% mean score        error.    -   Run 4. On 100 tests, 100(100.0%) matches and 0.81% mean score        error.    -   Run 5. On 100 tests, 100(100.0%) matches and 0.94% mean score        error.    -   Run 6. On 100 tests, 100(100.0%) matches and 0.82% mean score        error.    -   Run 7. On 100 tests, 100(100.0%) matches and 0.78% mean score        error.    -   Run 8. On 100 tests, 100(100.0%) matches and 0.86% mean score        error.    -   Run 9. On 100 tests, 100(100.0%) matches and 0.85% mean score        error.    -   Run 10. On 100 tests, 99(99.0%) matches and 0.86% mean score        error.    -   Average percentage of time LPHDR (with final DP correction)        finds nearest example=99.90%.    -   Average score error between LPHDR and DP=0.85%.

The “mean score error” values are considered below in the discussion ofweighted scores. The “matches” information is relevant here.

Of the ten runs, only one had any test, of the 100 tests performed,which yielded a nearest neighbor different from what the usual highprecision method yielded. Thus, the average percentage of matchesbetween the enhanced algorithm running with “fp+noise” arithmetic andthe usual method was 99.9%.

A similar computation was then performed using “Ins” arithmetic. In thiscase, the results were:

-   -   & ./a.out 5 10 1000000 100 0    -   Representation is LNS without noise.    -   Run 1. On 100 tests, 100(100.0%) matches and 0.15% mean score        error.    -   Run 2. On 100 tests, 100(100.0%) matches and 0.07% mean score        error.    -   Run 3. On 100 tests, 100(100.0%) matches and 0.08% mean score        error.    -   Run 4. On 100 tests, 100(100.0%) matches and 0.09% mean score        error.    -   Run 5. On 100 tests, 100(100.0%) matches and 0.11% mean score        error.    -   Run 6. On 100 tests, 100(100.0%) matches and 0.16% mean score        error.    -   Run 7. On 100 tests, 100(100.0%) matches and 0.07% mean score        error.    -   Run 8. On 100 tests, 100(100.0%) matches and 0.13% mean score        error.    -   Run 9. On 100 tests, 99(99.0%) matches and 0.17% mean score        error.    -   Run 10. On 100 tests, 98(98.0%) matches and 0.16% mean score        error.    -   Average percentage of time LPHDR (with final DP correction)        finds nearest example=99.70%.    -   Average score error between LPHDR and DP=0.12%.

The average percentage of matches was 99.7%, slightly worse than for“fp+noise”.

The accuracy shown by the enhanced nearest neighbor algorithm using twoforms of LPHDR arithmetic is surprising. To perform many calculationssequentially with 1% error and yet produce a final result with less than1% error may seem counter-intuitive. Nonetheless, the LPHDR arithmeticproves effective, and the accuracy shown is high enough to be useful inapplications for which approximate nearest neighbor calculations areuseful.

As an extreme case, a variant of fp+noise was tested in which the noisevaried uniformly from +10% to −5%. Thus, each arithmetic operationproduced a result that was between 10% too large and 5% too small. Theenhanced nearest neighbor algorithm, as described above, was performedwhere each run generated 100,000 Example vectors. The surprisingresults, below, show that even with this extreme level of imprecise,noisy, and non-zero mean LPHDR arithmetic, useful results can beachieved.

-   -   Run 1. On 100 tests, 97(97.0%) matches.    -   Run 2. On 100 tests, 100(100.0%) matches.    -   Run 3. On 100 tests, 100(100.0%) matches.    -   Run 4. On 100 tests, 98(98.0%) matches.    -   Run 5. On 100 tests, 98(98.0%) matches.    -   Run 6. On 100 tests, 99(99.0%) matches.    -   Run 7. On 100 tests, 99(99.0%) matches.    -   Run 8. On 100 tests, 99(99.0%) matches.    -   Run 9. On 100 tests, 99(99.0%) matches.    -   Run 10. On 100 tests, 99(99.0%) matches.    -   Average percentage of time LPHDR (with final DP correction)        finds nearest example=98.80%.

Efficiency.

In contrast to the surprising Accuracy results, it is clear to thosehaving ordinary skill in the art that the calculations of the enhancednearest neighbor algorithm can be performed efficiently in the computingmodel presented, where the arithmetic/memory units are connected in atwo-dimensional physical layout, using only local communication betweenPEs. However, this does not address the matter of keeping the machinebusy doing useful work using only low bandwidth to the host machine.

When computing the nearest neighbor to a single Test, the Test flowsacross all the PEs in the array. As discussed above, if the array is anM×M grid, it takes at least O(M) steps for the Test to pass through themachine and return results to the host. During this time the machineperforms 0(M×M) nearest neighbor distance computations, but since themachine is capable of performing 0(M×M) calculations at each step, afactor of O(M) is lost.

This speedup, compared to a serial machine, of a factor of O(M) issignificant and useful. However, the efficiency can be even higher. Ifsufficiently many Test vectors, say O(M), or more, are to be processedthen they can be streamed into the machine and made to flow through in apipelined fashion. The time to process O(M) Tests remains O(M), the sameas for a single Test, but now the machine performs O(M)×O(M×M) distancecomputations, and thus within a constant factor the full computingcapacity of the machine is used.

Thus, the machine is especially efficient if it is processing at leastas many Test vectors as the square root of the number of PEs. There areapplications that fit well into this form, such as pattern recognitionor compression of many independent Tests (e.g., blocks of an image,parts of a file, price histories of independent stocks) as well as theproblem of finding the nearest neighbor to every Example in the set ofExamples. This is in contrast to the general view among those havingordinary skill in the art, as discussed above, that machines with verymany arithmetic processing elements on a single chip, or similar, arenot very useful.

Application 2 Distance Weighted Scoring

A task related to Nearest Neighbor is Distance Weighted Scoring. In thistask, each Example has an associated Score. This is a number that insome way characterizes the Example. For instance, if the Examples areabstractions of the history of prices of a given stock, the Scores mightbe historical probabilities of whether the price is about to increase ordecrease. Given a Test vector, the task is to form a weighted sum of theScores of all the Examples, where the weights are a diminishing functionof the distance from the Test to the respective Examples. For example,this weighted score might be taken as a prediction of the future priceof the stock whose history is represented by the Test. This use ofembodiments of the invention might help support, for instance, highspeed trading of stocks, as is performed by certain “quantitative” hedgefunds, despite the general view by those having ordinary skill in theart that low precision computation is not of use in financialapplications.

The C code described above computes weighted scores along with nearestneighbors. The scores assigned to Examples in this computation arerandom numbers drawn uniformly from the range [0,1]. The weight for eachExample in this computation is defined to be the un-normalized weightfor the Example divided by the sum of the un-normalized weights for allExamples, where the un-normalized weight for each Example is defined tobe the reciprocal of the sum of one plus the squared distance from theExample to the Test vector. As discussed above, the code performs anumber of runs, each producing many Examples and Tests, and comparesresults of traditional floating point computations with resultscalculated using fp+noise and Ins arithmetic.

Looking again at the trace results of running the simulation, above, wesee that for fp+noise the LPHDR weighted scores on average were within0.85% of the correct value and never were as much as 1% different. ForIns arithmetic the errors were even smaller, averaging just 0.12% error.

These results are surprising given that computing an overall weightedscore involves summing the individual weighted scores associated witheach Example. Since each run was processing 1,000,000 Examples, thismeans that the sums were over one million small positive values. Thenaive method of summing one million small values with errors of about 1%in each addition should yield results that approximate noise. However,the code performs its sums using a long known method invented by Kahan(Kahan, William (January 1965), “Further remarks on reducing truncationerrors”, Communications of the ACM 8 (1): 40). The method makes itfeasible to perform long sums, such as are done for Distance WeightedScores, or as might be used in computational finance when computingprices of derivative securities using Monte Carlo methods, or forperforming deconvolution in image processing algorithms, as will bediscussed next.

The Efficiency of this algorithm is similar to that of NN, as discussedearlier. If many Test vectors are processed at once, the machineperforms especially efficiently.

Application 3 Removing Motion Blur in Images

In order to gather sufficient light to form an image, camera shuttersare often left open for long enough that camera motion can causeblurring. This can happen as a result of camera shake in inexpensiveconsumer cameras as well as with very expensive but fast moving camerasmounted on satellites or aircraft. If the motion path of the camera isknown (or can be computed) then the blur can be substantially removedusing various deblurring algorithms. One such algorithm is theRichardson-Lucy method (“RL”), and we show here that embodiments of thepresent invention can run that algorithm and produce useful results.Following the discussion format above, we discuss criteria of Accuracyand Efficiency.

Algorithm.

The Richardson-Lucy algorithm is well known and widely available. Assumethat an image has been blurred using a known kernel. In particular,assume that the kernel is a straight line and that the image has beenoriented so that the blur has occurred purely in a horizontal direction.Consider the particular kernel for which the J′th pixel in each row ofthe blurred image is the uniformly weighted mean of pixels J throughJ+31 in the original unblurred image.

Accuracy.

We implemented in the C programming language a straightforward versionof the RL method that uses LPHDR arithmetic. The program reads a testimage, blurs it using the kernel discussed above, then deblurs it usingeither fp+noise or Ins arithmetic. The RL algorithm computes sums, suchas when convolving the kernel with the current approximation of thedeblurred image. Our implementation computes these sums using the Kahanmethod, discussed earlier. FIG. 7 shows the test image in original form.It is a satellite picture of a building used during Barack Obama'sinauguration. FIG. 8 shows the image extremely blurred by the kernel. Itis difficult to see any particular objects in this image. FIG. 9 showsthe result of deblurring using standard floating point arithmetic. FIG.10 shows the result of deblurring using fp+noise arithmetic, and FIG. 11shows the result of deblurring using Ins arithmetic. In all these casesthe image is sufficiently restored that it is possible to recognizebuildings, streets, parking lots, and cars.

In addition to displaying the images herein for judgement using thehuman eye, we computed a numerical measure of deblurring performance. Wecomputed the mean difference, over all pixels in the image, between eachoriginal pixel value (a gray scale value from 0 to 255) and thecorresponding value in the image reconstructed by the RL method. Thosenumerical measures are shown below in Table 1:

TABLE 1 Image type Mean pixel error Blurred 32.0 RL using standardfloating point 13.0 RL using fp + noise 13.8 RL using Ins 14.8

These results, together with the subjective but important judgementsmade by the human eye, show that LPHDR arithmetic provides a substantialand useful degree of deblurring compared to standard floating pointarithmetic. Further, in this example we chose an extreme degree ofblurring, to better convey the concept and visual impact of thedeblurring using LPHDR arithmetic. On more gentle and typical blurkernels, the resulting deblurred images are much closer to the originalsthan in this case, as can be seen by shrinking the kernel length andrunning the RL algorithm with LPHDR arithmetic on those more typicalcases.

Efficiency.

It is clear to those with ordinary skill in the art that Richardson-Lucyusing a local kernel performs only local computational operations. Animage to be deblurred can be loaded into the PE array, storing one ormore pixels per PE, the deconvolution operation of RL can then beiterated dozens or hundreds of times, and the deblurred image can beread back to the host processor. As long as sufficient iterations areperformed, this makes efficient use of the machine.

An extreme form of image deblurring is the Iterative Reconstructionmethod used in computational tomography. Reconstructing 3D volumes from2D projections is an extremely computational task. The method discussedabove generalizes naturally to Iterative Reconstruction and makesefficient use of the machine.

Among the advantages of embodiments of the invention are one or more ofthe following.

PEs implemented according to certain embodiments of the presentinvention may be relatively small for PEs that can do arithmetic. Thismeans that there are many PEs per unit of resource (e.g., transistor,area, volume), which in turn means that there is a large amount ofarithmetic computational power per unit of resource. This enables largerproblems to be solved with a given amount of resource than doestraditional computer designs. For instance, a digital embodiment of thepresent invention built as a large silicon chip fabricated with currentstate of the art technology might perform tens of thousand of arithmeticoperations per cycle, as opposed to hundreds in a conventional GPU or ahandful in a conventional multicore CPU. These ratios reflect anarchitectural advantage of embodiments of the present invention thatshould persist as fabrication technology continues to improve, even aswe reach nanotechnology or other implementations for digital and analogcomputing.

Doing arithmetic with few resources generally means, and in theembodiments shown specifically means, that the arithmetic is done usinglow power. As a result, a machine implemented in accordance withembodiments of the present invention can have extremely high performancewith reasonable power (for instance in the tens of watts) or low power(for instance a fraction of a watt) with reasonably high performance.This means that such embodiments may be suitable for the full range ofcomputing, from supercomputers, through desktops, down to mobilecomputing. Similarly, and since cost is generally associated with theamount of available resources, embodiments of the present invention mayprovide a relatively high amount of computing power per unit of costcompared to conventional computing devices.

The SIMD architecture is rather old and is frequently discarded as anapproach to computer design by those having ordinary skill in the art.However, if the processing elements of a SIMD machine can be madeparticularly small while retaining important functionality, such asgeneral arithmetic ability, the architecture can be useful. Theembodiments presented herein have precisely those qualities.

The discovery that massive amounts of LPHDR arithmetic is useful as afairly general computing framework, as opposed to the common belief thatit is not useful, can be an advantage in any (massively ornon-massively) parallel machine design or non-parallel design, not justin SIMD embodiments. It could be used in FPGAs, FPAAs, GPU/SIMTmachines, MIMD machines, and in any kind of machine that uses compactarithmetic processing elements to perform large amounts of computationusing a small amount of resources (like transistors or volume).

Another advantage of embodiments of the present invention is that theyare not merely useful for performing computations efficiently ingeneral, but that they can be used to tackle a variety of real-worldproblems which are typically assumed to require high-precision computingelements, even though such embodiments include only (or predominantly)low-precision computing elements. Although several examples of suchreal-world problems have been presented herein, and although we havealso had success implementing non-bonded force field computations formolecular dynamics simulation and other tasks, these are merely examplesand do not constitute an exhaustive set of the real-world problems thatembodiments of the present invention may be used to solve.

The embodiments disclosed above are merely examples and do notconstitute limitations of the present invention. Rather, embodiments ofthe present invention may be implemented in a variety of other ways,such as the following.

For example, embodiments of the present invention may represent valuesin any of a variety of ways, such as by using digital or analogrepresentations, such as fixed point, logarithmic, or floating pointrepresentations, voltages, currents, charges, pulse width, pulsedensity, frequency, probability, spikes, timing, or combinationsthereof. These underlying representations may be used individually or incombination to represent the LPHDR values. LPHDR arithmetic circuits maybe implemented in any of a variety of ways, such as by using variousdigital methods (which may be parallel or serial, pipelined or not) oranalog methods or combinations thereof. Arithmetic elements may beconnected using various connection architectures, such as nearest 4,nearest 8, hops of varying degree, and architectures which may or maynot be rectangular or grid-like. Any method may be used forcommunication among arithmetic elements, such as parallel or serial,digital or analog or mixed-mode communication. Arithmetic elements mayoperate synchronously or asynchronously, and may operate globallysimultaneously or not. Arithmetic elements may be implemented, forexample, on a single physical device, such as a silicon chip, or spreadacross multiple devices and an embodiment built from multiple devicesmay have its arithmetic elements connected in a variety of ways,including for example being connected as a grid, torus, hypercube, tree,or other method. Arithmetic elements may be connected to a host machine,if any, in a variety of ways, depending on the cost and bandwidth andother requirements of a particular embodiment. For example there may bemany host machines connected to the collection of arithmetic elements.

Although certain embodiments of the present invention are described asbeing implemented as a SIMD architecture, this is merely an example anddoes not constitute a limitation of the present invention. For example,embodiments of the present invention may be implemented asreconfigurable architectures, such as but not limited to programmablelogic devices, field programmable analog arrays, or field programmablegate array architectures, such as a design in which existing multiplierblocks of an FPGA are replaced with or supplemented by LPHDR arithmeticelements of any of the kinds disclosed herein, or for example in whichLPHDR elements are included in a new or existing reconfigurable devicedesign. As another example, embodiments of the present invention may beimplemented as a GPU or SIMT-style architecture which incorporates LPHDRarithmetic elements of any of the kinds disclosed herein. For example,LPHDR elements could supplement or replace traditional arithmeticelements in current or new graphics processing unit designs. As yetanother example, embodiments of the present invention may be implementedas a MIMD-style architecture which incorporates LPHDR arithmeticelements of any of the kinds disclosed herein. For example, LPHDRarithmetic elements could supplement or replace traditional arithmeticelements in current or new MIMD computing system designs. As yet anotherexample, embodiments of the present invention may be implemented as anykind of machine, including a massively parallel machine, which usescompact arithmetic processing elements to provide large amounts ofarithmetic computing capability using a small amount of resources (forexample, transistors or area or volume) compared with traditionalarchitectures.

Although certain embodiments of the present invention are describedherein as executing software, this is merely an example and does notconstitute a limitation of the present invention. Alternatively, forexample, embodiments of the present invention may be implemented usingmicrocode or a hardware sequencer or state machine or other controllerto control LPHDR arithmetic elements of any of the kinds disclosedherein. Alternatively, for example, embodiments of the present inventionmay be implemented using hardwired, burned, or otherwise pre-programmedcontrollers to control LPHDR arithmetic elements of any of the kindsdisclosed herein.

Although certain embodiments of the present invention are describedherein as being implemented using custom silicon as the hardware, thisis merely an example and does not constitute a limitation of the presentinvention. Alternatively, for example, embodiments of the presentinvention may be implemented using FPGA or other reconfigurable chips asthe underlying hardware, in which the FPGAs or other reconfigurablechips are configured to perform the LPHDR operations disclosed herein.As another example, embodiments of the present invention may beimplemented using any programmable conventional digital or analogcomputing architecture (including those which use high-precisioncomputing elements, including those which use other kinds of non-LPHDRhardware to perform LPHDR arithmetic, and including those which aremassively parallel) which has been programmed with software to performthe LPHDR operations disclosed herein. For example, embodiments of thepresent invention may be implemented using a software emulator of thefunctions disclosed herein.

As yet another example, embodiments of the present invention may beimplemented using 3D fabrication technologies, whether based on siliconchips or otherwise. Some example embodiments are those in which a memorychip has been bonded onto a processor or other device chip or in whichseveral memory and/or processor or other device chips have been bondedto each other in a stack. 3D embodiments of the present invention arevery useful as they may be denser than 2D embodiments and may enable 3Dcommunication of information between the processing units, which enablesmore algorithms to run efficiently on those embodiments compared to 2Dembodiments.

Although certain embodiments of the present invention are describedherein as being implemented using silicon chip fabrication technology,this is merely an example and does not constitute a limitation of thepresent invention. Alternatively, for example, embodiments of thepresent invention may be implemented using technologies that may enableother sorts of traditional digital and analog computing processors orother devices. Examples of such technologies include variousnanomechanical and nanoelectronic technologies, chemistry basedtechnologies such as for DNA computing, nanowire and nanotube basedtechnologies, optical technologies, mechanical technologies, biologicaltechnologies, and other technologies whether based on transistors or notthat are capable of implementing LPHDR architectures of the kindsdisclosed herein.

Certain embodiments of the present invention have been described as“massively parallel” embodiments. Although certain embodiments of thepresent invention may include thousands, millions, or more arithmeticunits, embodiments of the present invention may include any number ofarithmetic units (as few as one). For example, even an embodiment whichincludes only a single LPHDR unit may be used within a serial processingunit or other device to provide a significant amount of LPHDR processingpower in a small, inexpensive processor or other device.

For certain embodiments of the present invention, even if implementedusing only digital techniques, the arithmetic operations may not yielddeterministic, repeatable, or the most accurate possible results withinthe chosen low precision representation. For instance, on certainspecific input values, an arithmetic operation may produce a resultwhich is not the nearest value in the chosen representation to the truearithmetic result.

The degree of precision of a “low precision, high dynamic range”arithmetic element may vary from implementation to implementation. Forexample, in certain embodiments, a LPHDR arithmetic element producesresults which include fractions, that is, values greater than zero andless than one. For example, in certain embodiments, a LPHDR arithmeticelement produces results which are sometimes (or all of the time) nocloser than 0.05% to the correct result (that is, the absolute value ofthe difference between the produced result and the correct result is nomore than one-twentieth of one percent of the absolute value of thecorrect result). As another example, a LPHDR arithmetic element mayproduce results which are sometimes (or all of the time) no closer than0.1% to the correct result. As another example, a LPHDR arithmeticelement may produce results which are sometimes (or all of the time) nocloser than 0.2% to the correct result. As yet another example, a LPHDRarithmetic element may produce results which are sometimes (or all ofthe time) no closer than 0.5% to the correct result. As yet furtherexamples, a LPHDR arithmetic element may produce results which aresometimes (or all of the time) no closer than 1%, or 2%, or 5%, or 10%,or 20% to the correct result.

Besides having various possible degrees of precision, implementationsmay vary in the dynamic range of the space of values they process. Forexample, in certain embodiments, a LPHDR arithmetic element processesvalues in a space which may range approximately from one millionth toone million. As another example, in certain embodiments, a LPHDRarithmetic element processes values in a space which may rangeapproximately from one billionth to one billion. As yet another example,in certain embodiments, a LPHDR arithmetic element processes values in aspace which may range approximately from one sixty five thousandth tosixty five thousand. As yet further examples, in certain embodiments, aLPHDR arithmetic element processes values in a space which may rangefrom any specific value between zero and one sixty five thousandth up toany specific value greater than sixty five thousand. As yet furtherexamples, other embodiments may process values in spaces with dynamicranges that may combine and may fall between the prior examples, forexample ranging from approximately one billionth to ten million. In allof these example embodiments of the present invention, as well as inother embodiments, the values that we are discussing may be signed, sothat the above descriptions characterize the absolute values of thenumbers being discussed.

The frequency with which LPHDR arithmetic elements may yield onlyapproximations to correct results may vary from implementation toimplementation. For example, consider an embodiment in which LPHDRarithmetic elements can perform one or more operations (perhapsincluding, for example, trigonometric functions), and for each operationthe LPHDR elements each accept a set of inputs drawn from a range ofvalid values, and for each specific set of input values the LPHDRelements each produce one or more output values (for example,simultaneously computing both sin and cos of an input), and the outputvalues produced for a specific set of inputs may be deterministic ornon-deterministic. In such an example embodiment, consider further afraction F of the valid inputs and a relative error amount E by whichthe result calculated by an LPHDR element may differ from themathematically correct result. In certain embodiments of the presentinvention, for each LPHDR arithmetic element, for at least one operationthat the LPHDR unit is capable of performing, for at least fraction F ofthe possible valid inputs to that operation, for at least one outputsignal produced by that operation, the statistical mean, over repeatedexecution, of the numerical values represented by that output signal ofthe LPHDR unit, when executing that operation on each of thoserespective inputs, differs by at least E from the result of an exactmathematical calculation of the operation on those same input values,where F is 1% and E is 0.05%. In several other example embodiments, F isnot 1% but instead is one of 2%, or 5%, or 10%, or 20%, or 50%. For eachof these example embodiments, each with some specific value for F, thereare other example embodiments in which E is not 0.05% but instead is0.1%, or 0.2%, or 0.5%, or 1%, or 2%, or 5%, or 10%, or 20%. Thesevaried embodiments are merely examples and do not constitute limitationsof the present invention.

For certain devices (such as computers or processors or other devices)embodied according the present invention, the number of LPHDR arithmeticelements in the device (e.g., computer or processor or other device)exceeds the number, possibly zero, of arithmetic elements in the devicewhich are designed to perform high dynamic range arithmetic oftraditional precision (that is, floating point arithmetic with a wordlength of 32 or more bits). If NL is the total number of LPHDR elementsin such a device, and NH is the total number of elements in the devicewhich are designed to perform high dynamic range arithmetic oftraditional precision, then NL exceeds T(NH), where T( ) is somefunction. Any of a variety of functions may be used as the function T(). For example, in certain embodiments, T(NH) may be twenty plus threetimes NH, and the number of LPHDR arithmetic elements in the device mayexceed twenty more than three times the number of arithmetic elements inthe device, if any, designed to perform high dynamic range arithmetic oftraditional precision. As another example, in certain embodiments, thenumber of LPHDR arithmetic elements in the device may exceed fifty morethan five times the number of arithmetic elements in the device, if any,designed to perform high dynamic range arithmetic of traditionalprecision. As yet another example, in certain embodiments, the number ofLPHDR arithmetic elements in the device may exceed one hundred more thanfive times the number of arithmetic elements in the device, if any,designed to perform high dynamic range arithmetic of traditionalprecision. As yet another example, in certain embodiments, the number ofLPHDR arithmetic elements in the device may exceed one thousand morethan five times the number of arithmetic elements in the device, if any,designed to perform high dynamic range arithmetic of traditionalprecision. As yet another example, in certain embodiments, the number ofLPHDR arithmetic elements in the device may exceed five thousand morethan five times the number of arithmetic elements in the device, if any,designed to perform high dynamic range arithmetic of traditionalprecision. Certain embodiments of the present invention may beimplemented within a single physical device, such as but not limited toa silicon chip or a chip stack or a chip package or a circuit board, andthe number NL of LPHDR elements in the physical device and the number NHof elements designed to perform high dynamic range arithmetic oftraditional precision in the physical device may be the total counts ofthe respective elements within that physical device. Certain embodimentsof the present invention may be implemented in a computing systemincluding more than one physical device, such as but not limited to acollection of silicon chips or chip stacks or chip packages or circuitboards coupled to and communicating with each other using any means(such as a bus, switch, any kind of network connection, or other meansof communication), and in this case the number NL of LPHDR elements inthe computing system and the number NH of elements designed to performhigh dynamic range arithmetic of traditional precision in the computingsystem may be the total counts of the respective elements within allthose physical devices jointly.

Certain embodiments of the present invention may constitute, or may bepart of, processors, which are devices capable of executing software toperform computations. Such processors may include mechanisms for storingsoftware, for using the software to determine what operations toperform, for performing those operations, for storing numerical data,for modifying data according to the software specified operations, andfor communicating with devices connected to the processor. Processorsmay be reconfigurable devices, such as, without limitation, fieldprogrammable arrays. Processors may be co-processors to assist hostmachines or may be capable of operating independently of an externalhost. Processors may be formed as a collection of component hostprocessors and co-processors of various types, such as CPUs, GPUs,FPGAs, or other processors or other devices, which in the art may bereferred to as a heterogeneous processor design or heterogeneouscomputing system, some or all of which components might incorporate thesame or distinct varieties of embodiments of the present invention.

Embodiments of the present invention may, however, be implemented indevices in addition to or other than processors. For example, a computerincluding a processor and other components (such as memory coupled tothe processor by a data path), wherein the processor includes componentsfor performing LPHDR operations in any of the ways disclosed herein, isan example of an embodiment of the present invention. More generally,any device or combination of devices, whether or not falling within themeaning of a “processor,” which performs the functions disclosed hereinmay constitute an example of an embodiment of the present invention.

More generally, any of the techniques described above may beimplemented, for example, in hardware, software tangibly stored on acomputer-readable medium, firmware, or any combination thereof. Thetechniques described above may be implemented in one or more computerprograms executing on a programmable computer including a processor, astorage medium readable by the processor (including, for example,volatile and non-volatile memory and/or storage elements), at least oneinput device, and at least one output device. Program code may beapplied to input entered using the input device to perform the functionsdescribed and to generate output. The output may be provided to one ormore output devices.

Each computer program within the scope of the claims below may beimplemented in any programming language, such as assembly language,machine language, a high-level procedural programming language, or anobject-oriented programming language. The programming language may, forexample, be a compiled or interpreted programming language.

Each such computer program may be implemented in a computer programproduct tangibly embodied in a machine-readable storage device forexecution by a computer processor. Method steps of the invention may beperformed by a computer processor executing a program tangibly embodiedon a computer-readable medium to perform functions of the invention byoperating on input and generating output. Suitable processors include,by way of example, both general and special purpose microprocessors.Generally, the processor receives instructions and data from a read-onlymemory and/or a random access memory. Storage devices suitable fortangibly embodying computer program instructions include, for example,all forms of non-volatile memory, such as semiconductor memory devices,including EPROM, EEPROM, and flash memory devices; magnetic disks suchas internal hard disks and removable disks; magneto-optical disks; andCD-ROMs. Any of the foregoing may be supplemented by, or incorporatedin, specially-designed ASICs (application-specific integrated circuits)or FPGAs (Field-Programmable Gate Arrays). A computer can generally alsoreceive programs and data from a storage medium such as an internal disk(not shown) or a removable disk. These elements will also be found in aconventional desktop or workstation computer as well as other computerssuitable for executing computer programs implementing the methodsdescribed herein, which may be used in conjunction with any digitalprint engine or marking engine, display monitor, or other raster outputdevice capable of producing color or gray scale pixels on paper, film,display screen, or other output medium.

1. A device comprising: at least one first low precision high dynamicrange (LPHDR) execution unit adapted to execute a first operation on afirst input signal representing a first numerical value to produce afirst output signal representing a second numerical value, wherein thedynamic range of the possible valid inputs to the first operation is atleast as wide as from 1/65,000 through 65,000 and for at least X=5% ofthe possible valid inputs to the first operation, the statistical mean,over repeated execution of the first operation on each specific inputfrom the at least X % of the possible valid inputs to the firstoperation, of the numerical values represented by the first outputsignal of the LPHDR unit executing the first operation on that inputdiffers by at least Y=0.05% from the result of an exact mathematicalcalculation of the first operation on the numerical values of that sameinput; and at least one first computing device adapted to control theoperation of the at least one first LPHDR execution unit.
 2. The methodof claim 1, wherein the at least one first computing device comprises atleast one of a central processing unit (CPU), a graphics processing unit(GPU), a field programmable gate array (FPGA), a microcode-basedprocessor, a hardware sequencer, and a state machine.
 3. The device ofclaim 2, wherein the number of LPHDR execution units in the deviceexceeds by at least one hundred the non-negative integer number ofexecution units in the device adapted to execute at least the operationof multiplication on floating point numbers that are at least 32 bitswide.
 4. The device of claim 3, wherein X=10%.
 5. The device of claim 3,wherein Y=0.2%.
 6. The device of claim 3, wherein X=10% and Y=0.2%. 7.The device of claim 3, wherein the dynamic range of the possible validinputs to the first operation is at least as wide as from 1/1,000,000through 1,000,000.
 8. The device of claim 3, wherein the first operationis multiplication.
 9. A device comprising: at least one first lowprecision high dynamic range (LPHDR) execution unit adapted to execute afirst operation on a first input signal representing a first numericalvalue to produce a first output signal representing a second numericalvalue, wherein the dynamic range of the possible valid inputs to thefirst operation is at least as wide as from 1/65,000 through 65,000 andfor at least X=5% of the possible valid inputs to the first operation,the statistical mean, over repeated execution of the first operation oneach specific input from the at least X % of the possible valid inputsto the first operation, of the numerical values represented by the firstoutput signal of the LPHDR unit executing the first operation on thatinput differs by at least Y=0.05% from the result of an exactmathematical calculation of the first operation on the numerical valuesof that same input; and wherein the at least one first LPHDR executionunit comprises at least one of a chemistry-technology-based executionunit, a biological-technology-based execution unit, aDNA-technology-based execution unit, a nanomechanical-technology-basedexecution unit, a nanoelectronic-technology-based execution unit, ananowire-technology-based execution unit, a nanotube-technology-basedexecution unit, and an optical-technology-based execution unit.
 10. Thedevice of claim 9, wherein the number of LPHDR execution units in thedevice exceeds by at least one hundred the non-negative integer numberof execution units in the device adapted to execute at least theoperation of multiplication on floating point numbers that are at least32 bits wide.
 11. The device of claim 10, wherein X=10%.
 12. The deviceof claim 10, wherein Y=0.2%.
 13. The device of claim 10, wherein X=10%and Y=0.2%.
 14. The device of claim 10, wherein the dynamic range of thepossible valid inputs to the first operation is at least as wide as from1/1,000,000 through 1,000,000.
 15. The device of claim 10, wherein thefirst operation is multiplication.
 16. A device comprising: a pluralityof components comprising: at least one first low precision high dynamicrange (LPHDR) execution unit adapted to execute a first operation on afirst input signal representing a first numerical value to produce afirst output signal representing a second numerical value, wherein thedynamic range of the possible valid inputs to the first operation is atleast as wide as from 1/65,000 through 65,000 and for at least X=5% ofthe possible valid inputs to the first operation, the statistical mean,over repeated execution of the first operation on each specific inputfrom the at least X % of the possible valid inputs to the firstoperation, of the numerical values represented by the first outputsignal of the LPHDR unit executing the first operation on that inputdiffers by at least Y=0.05% from the result of an exact mathematicalcalculation of the first operation on the numerical values of that sameinput.
 17. The device of claim 16, wherein the plurality of componentsare arranged in a stack.
 18. The device of claim 17, wherein the numberof LPHDR execution units in the device exceeds by at least one hundredthe non-negative integer number of execution units in the device adaptedto execute at least the operation of multiplication on floating pointnumbers that are at least 32 bits wide.
 19. The device of claim 18,wherein X=10%.
 20. The device of claim 18, wherein Y=0.2%.
 21. Thedevice of claim 18, wherein X=10% and Y=0.2%.
 22. The device of claim18, wherein the dynamic range of the possible valid inputs to the firstoperation is at least as wide as from 1/1,000,000 through 1,000,000. 23.The device of claim 18, wherein the first operation is multiplication.24. The device of claim 16, wherein the plurality of components arebonded.
 25. The device of claim 16, wherein the plurality of componentsare arranged in a stack and bonded.
 26. A device comprising a computerprocessor and a computer-readable memory storing computer programinstructions, wherein the computer program instructions are executableby the processor to emulate a second device comprising: at least onefirst low precision high-dynamic range (LPHDR) execution unit adapted toexecute a first operation on a first input signal representing a firstnumerical value to produce a first output signal representing a secondnumerical value; wherein the dynamic range of the possible valid inputsto the first operation is at least as wide as from 1/65,000 through65,000 and for at least X=5% of the possible valid inputs to the firstoperation, the statistical mean, over repeated execution of the firstoperation on each specific input from the at least X % of the possiblevalid inputs to the first operation, of the numerical values representedby the first output signal of the LPHDR unit executing the firstoperation on that input differs by at least Y=0.05% from the result ofan exact mathematical calculation of the first operation on thenumerical values of that same input; and wherein the at least one firstLPHDR execution unit comprises at least one of achemistry-technology-based execution unit, a biological-technology-basedexecution unit, a DNA-technology-based execution unit, ananomechanical-technology-based execution unit, ananoelectronic-technology-based execution unit, ananowire-technology-based execution unit, a nanotube-technology-basedexecution unit, and an optical-technology-based execution unit.
 27. Thedevice of claim 26, wherein the number of LPHDR execution units in thesecond device exceeds by at least one hundred the non-negative integernumber of execution units in the second device adapted to execute atleast the operation of multiplication on floating point numbers that areat least 32 bits wide.
 28. The device of claim 27, wherein X=10%. 29.The device of claim 27, wherein Y=0.2%.
 30. The device of claim 27,wherein X=10% and Y=0.2%.
 31. The device of claim 27, wherein thedynamic range of the possible valid inputs to the first operation is atleast as wide as from 1/1,000,000 through 1,000,000.
 32. The device ofclaim 27, wherein the first operation is multiplication.
 33. A devicecomprising a computer processor and a computer-readable memory storingcomputer program instructions, wherein the computer program instructionsare executable by the processor to emulate a second device comprising: aplurality of components comprising: at least one first low precisionhigh-dynamic range (LPHDR) execution unit adapted to execute a firstoperation on a first input signal representing a first numerical valueto produce a first output signal representing a second numerical value;wherein the dynamic range of the possible valid inputs to the firstoperation is at least as wide as from 1/65,000 through 65,000 and for atleast X=5% of the possible valid inputs to the first operation, thestatistical mean, over repeated execution of the first operation on eachspecific input from the at least X % of the possible valid inputs to thefirst operation, of the numerical values represented by the first outputsignal of the LPHDR unit executing the first operation on that inputdiffers by at least Y=0.05% from the result of an exact mathematicalcalculation of the first operation on the numerical values of that sameinput.
 34. The device of claim 33, wherein the plurality of componentsare arranged in a stack.
 35. The device of claim 34, wherein the numberof LPHDR execution units in the second device exceeds by at least onehundred the non-negative integer number of execution units in the seconddevice adapted to execute at least the operation of multiplication onfloating point numbers that are at least 32 bits wide.
 36. The device ofclaim 35, wherein X=10%.
 37. The device of claim 35, wherein Y=0.2%. 38.The device of claim 35, wherein X=10% and Y=0.2%.
 39. The device ofclaim 35, wherein the dynamic range of the possible valid inputs to thefirst operation is at least as wide as from 1/1,000,000 through1,000,000.
 40. The device of claim 35, wherein the first operation ismultiplication.
 41. The device of claim 33, wherein the plurality ofcomponents are bonded.
 42. The device of claim 33, wherein the pluralityof components are arranged in a stack and bonded.